Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(perception_online_evaluator): publish metrics of each object class #6556

Merged
merged 1 commit into from
Mar 7, 2024
Merged

feat(perception_online_evaluator): publish metrics of each object class #6556

merged 1 commit into from
Mar 7, 2024

Conversation

kosuke55
Copy link
Contributor

@kosuke55 kosuke55 commented Mar 6, 2024 •
edited
Loading

Description

publish metrics of each object class
e.g. predicted_path_deviation_CAR_5.00

Tests performed

psim

image

evaluator_description: feat/perception_eval_object_class
2024/03/06 https://evaluation.tier4.jp/evaluation/reports/0d75e8bd-369e-5045-b438-8801d9012492/?project_id=prd_jt

unit test

1: [==========] Running 16 tests from 1 test suite.
1: [----------] Global test environment set-up.
1: [----------] 16 tests from EvalTest
1: [ RUN      ] EvalTest.testLateralDeviation_deviation0
1: [       OK ] EvalTest.testLateralDeviation_deviation0 (1627 ms)
1: [ RUN      ] EvalTest.testLateralDeviation_deviation1
1: [       OK ] EvalTest.testLateralDeviation_deviation1 (1625 ms)
1: [ RUN      ] EvalTest.testLateralDeviation_oscillation
1: [       OK ] EvalTest.testLateralDeviation_oscillation (2635 ms)
1: [ RUN      ] EvalTest.testLateralDeviation_distortion
1: [       OK ] EvalTest.testLateralDeviation_distortion (2654 ms)
1: [ RUN      ] EvalTest.testLateralDeviation_deviation0_PEDESTRIAN
1: [       OK ] EvalTest.testLateralDeviation_deviation0_PEDESTRIAN (1621 ms)
1: [ RUN      ] EvalTest.testYawDeviation_deviation0
1: [       OK ] EvalTest.testYawDeviation_deviation0 (1643 ms)
1: [ RUN      ] EvalTest.testYawDeviation_deviation1
1: [       OK ] EvalTest.testYawDeviation_deviation1 (1620 ms)
1: [ RUN      ] EvalTest.testYawDeviation_oscillation
1: [       OK ] EvalTest.testYawDeviation_oscillation (2641 ms)
1: [ RUN      ] EvalTest.testYawDeviation_distortion
1: [       OK ] EvalTest.testYawDeviation_distortion (2637 ms)
1: [ RUN      ] EvalTest.testYawDeviation_oscillation_rotate
1: [       OK ] EvalTest.testYawDeviation_oscillation_rotate (2642 ms)
1: [ RUN      ] EvalTest.testYawDeviation_distortion_rotate
1: [       OK ] EvalTest.testYawDeviation_distortion_rotate (2635 ms)
1: [ RUN      ] EvalTest.testYawDeviation_deviation0_PEDESTRIAN
1: [       OK ] EvalTest.testYawDeviation_deviation0_PEDESTRIAN (1623 ms)
1: [ RUN      ] EvalTest.testPredictedPathDeviation_deviation0
1: [       OK ] EvalTest.testPredictedPathDeviation_deviation0 (1630 ms)
1: [ RUN      ] EvalTest.testPredictedPathDeviation_deviation1
1: [       OK ] EvalTest.testPredictedPathDeviation_deviation1 (1624 ms)
1: [ RUN      ] EvalTest.testPredictedPathDeviation_deviation2
1: [       OK ] EvalTest.testPredictedPathDeviation_deviation2 (1622 ms)
1: [ RUN      ] EvalTest.testPredictedPathDeviation_deviation0_PEDESTRIAN
1: [       OK ] EvalTest.testPredictedPathDeviation_deviation0_PEDESTRIAN (1620 ms)
1: [----------] 16 tests from EvalTest (32100 ms total)
1:
1: [----------] Global test environment tear-down
1: [==========] 16 tests from 1 test suite ran. (32100 ms total)
1: [  PASSED  ] 16 tests.

Effects on system behavior

Not applicable.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@github-actions github-actions bot added the component:evaluator Evaluation tools for planning, localization etc. (auto-assigned) label Mar 6, 2024
@kosuke55 kosuke55 added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Mar 6, 2024
@kosuke55 kosuke55 force-pushed the feat/perception_eval_object_class branch 2 times, most recently from 349b536 to 5dd8c49 Compare March 6, 2024 14:28
@kosuke55 kosuke55 force-pushed the feat/perception_eval_object_class branch from 5dd8c49 to 713dfec Compare March 6, 2024 14:28
@codecov Codecov
Copy link

codecov bot commented Mar 6, 2024

Codecov Report

Attention: Patch coverage is 42.10526% with 55 lines in your changes are missing coverage. Please review.

Project coverage is 14.80%. Comparing base (59f7031) to head (713dfec).
Report is 11 commits behind head on main.

Files Patch % Lines
...tor/test/test_perception_online_evaluator_node.cpp 47.16% 0 Missing and 28 partials ⚠️
...eption_online_evaluator/src/metrics_calculator.cpp 32.43% 2 Missing and 23 partials ⚠️
...n_online_evaluator/src/utils/objects_filtering.cpp 60.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #6556      +/-   ##
==========================================
+ Coverage   14.78%   14.80%   +0.01%     
==========================================
  Files        1917     1917              
  Lines      132038   132078      +40     
  Branches    39228    39245      +17     
==========================================
+ Hits        19523    19555      +32     
+ Misses      90726    90711      -15     
- Partials    21789    21812      +23
Flag Coverage Δ *Carryforward flag
differential 47.86% <42.10%> (?)
total 14.78% <ø> (-0.01%) ⬇️ Carriedforward from 59f7031

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

rej55
rej55 approved these changes Mar 7, 2024
View reviewed changes
Copy link
Contributor

@rej55 rej55 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rej55 rej55 merged commit 7081a61 into autowarefoundation:main Mar 7, 2024
24 checks passed
@rej55 rej55 deleted the feat/perception_eval_object_class branch March 7, 2024 02:57
@hayato-m126 hayato-m126 mentioned this pull request Mar 7, 2024
Closed
HansRobo pushed a commit that referenced this pull request Mar 12, 2024
@kosuke55 @HansRobo
feat(perception_online_evaluator): publish metrics of each object cla…
0bfa5e1 
…ss (#6556)

Signed-off-by: kosuke55 <[email protected]>
Signed-off-by: Kotaro Yoshimoto <[email protected]>
kaigohirao pushed a commit to kaigohirao/autoware.universe that referenced this pull request Mar 22, 2024
@kosuke55 @kaigohirao
feat(perception_online_evaluator): publish metrics of each object cla…
885c1fd 
…ss (autowarefoundation#6556)

Signed-off-by: kosuke55 <[email protected]>
Signed-off-by: kaigohirao <[email protected]>
badai-nguyen pushed a commit to tier4/autoware.universe that referenced this pull request Apr 23, 2024
@kosuke55 @badai-nguyen
feat(perception_online_evaluator): publish metrics of each object cla…
5a83c51 
…ss (autowarefoundation#6556)

Signed-off-by: kosuke55 <[email protected]>
badai-nguyen pushed a commit to tier4/autoware.universe that referenced this pull request Apr 23, 2024
@kosuke55 @badai-nguyen
feat(perception_online_evaluator): publish metrics of each object cla…
bcabfdc 
…ss (autowarefoundation#6556)

Signed-off-by: kosuke55 <[email protected]>
badai-nguyen pushed a commit to tier4/autoware.universe that referenced this pull request May 9, 2024
@kosuke55 @badai-nguyen
feat(perception_online_evaluator): publish metrics of each object cla…
20b88b7 
…ss (autowarefoundation#6556)

Signed-off-by: kosuke55 <[email protected]>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
@kosuke55 @karishma1911
feat(perception_online_evaluator): publish metrics of each object cla…
3aa14ba 
…ss (autowarefoundation#6556)

Signed-off-by: kosuke55 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rej55 rej55 rej55 approved these changes

@ktro2828 ktro2828 Awaiting requested review from ktro2828 ktro2828 is a code owner

@kyoichi-sugahara kyoichi-sugahara Awaiting requested review from kyoichi-sugahara kyoichi-sugahara is a code owner

@miursh miursh Awaiting requested review from miursh miursh is a code owner

@YoshiRi YoshiRi Awaiting requested review from YoshiRi YoshiRi is a code owner

Assignees

@rej55 rej55

Labels
component:evaluator Evaluation tools for planning, localization etc. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kosuke55 @rej55